Bag-Based Relevance Feedback Framework for Large-Scale CBIR

Relevance feedback has been used to effectively improve the retrieval performance of CBIR. In this paper, we propose a bag-based relevance feedback framework for largescale CBIR. We use kmeans-based clustering method to partition the large-scale image database based on (noisy) textual and low-level visual features. When the user selects relevant and irrelevant images as their feedback, we introduce a two-view consensus approach to automatically find a cluster in which there are at least a portion of relevant images (referred to as a “positive bag”), and a cluster in which there is almost no relevant images (referred to as a “negative bag”). By treating these clusters as “bags” and images in the clusters as “instances”, we formulate this problem as a multiple-instance (MI) learning problem. Thus, we can apply MI learning methods such as mi-SVM to enhance the retrieval performance. In addition, we develop a Generalized Multiple-Instance SVM (GMI-SVM) method to further enhance the retrieval by propagating the labels from the bag level to the instance level. Moreover, with the user’s weak annotation of bags, our bag-based relevance feedback framework can significantly improve the precision. Comprehensive experiments on the challenging dataset NUS-WIDE, clearly demonstrate the effectiveness of our bag-based RF framework for large-scale image retrieval.